feat: Add schema construction DSL #569
Conversation
paleolimbot
left a comment
paleolimbot
left a comment
There was a problem hiding this comment.
Just some initial thoughts! If we're really headed for readable schema construction, Arrow C++-like constructors (e.g., int32("field name")) are more likely to be what we want (and might be substantially less complicated to implement). I probably would have gone towards templating on ArrowType and using our C library constructors instead of introducing another way to create schemas.
It's worth looking at how https://github.com/man-group/sparrow does this (I think they have something pretty similar to construct types/schemas).
I was hoping to merge #561 before adding anything to the testing library (it could use a review!).
|
|
||
| /// @} | ||
|
|
||
| namespace dsl { |
| /// \brief An alias to express a sequence of key value pairs. | ||
| /// | ||
| /// Each pair is a string formatted like "key=value". | ||
| using metadata = std::vector<std::string>; |
There was a problem hiding this comment.
Would std::vector<std::pair<std::string, std::string>> work? I'm not sure the world is clamouring to create metadata keys that contain the = symbol but it seems like it would be nice to avoid string parsing if we can.
| schema{children{ | ||
| {"i", "int32 field name", | ||
| metadata{ | ||
| "some_key=some_value", | ||
| }}, | ||
| {"i", dictionary{"u"}, "dictionary field name", | ||
| metadata{ | ||
| "some_key=some_value", | ||
| }, | ||
| ARROW_FLAG_NULLABLE}, | ||
| }}; |
There was a problem hiding this comment.
In tests I think I would still prefer to see the C calls (but perhaps this is not where you were headed with this)
2 participants
Construct
nanoarrow::UniqueSchemas with a tiny C++ DSL:schema{children{ {"i", "i32", metadata{ "some_key=some_value", }}, {"i", "i32", dictionary{{"u"}}, metadata{ "some_key=some_value", }, ARROW_FLAG_NULLABLE}, }};The python-equivalent signature is approximately:
After the first two arguments, all the rest can be provided in any order.